An Effective Framework for Semantic Event Detection

We propose a two-step Event-Level Feature (ELF) learning framework for automatic detection of semantic events. In the first step an elementary-level feature is generated to represent images and videos. Then in the second step an ELF is constructed on top of the elementary features to model each event as a feature vector. Semantic event detectors can be built based on the ELF. Various ELFs are generated from different types of elementary-level features by using both cross-domain and within-domain learning: crossdomain approaches use three sets of concept scores at both image and region level that are learned from three external data sources; within-domain approaches use low-level visual features at both image and region level. Different types of ELFs complement each other for improved semantic event detection. Experiments over a large real consumer data set confirm significant improvements, e.g., over 90% MAP gain compared to the previous semantic event detection method.